Search for: All records

While Reinforcement learning (RL), especially Deep RL (DRL), has shown outstanding performance in video games, little evidence has shown that DRL can be successfully applied to human-centric tasks where the ultimate RL goal is to make the \textit{human-agent interactions} productive and fruitful. In real-life, complex, human-centric tasks, such as education and healthcare, data can be noisy and limited. Batch RL is designed for handling such situations where data is \textit{limited yet noisy}, and where \textit{building simulations is challenging}. In two consecutive empirical studies, we investigated Batch DRL for pedagogical policy induction, to choose student learning activities in an Intelligent Tutoring System. In Fall 2018 (F18), we compared the Batch DRL policy to an Expert policy, but found no significant difference between the DRL and Expert policies. In Spring 2019 (S19), we augmented the Batch DRL-induced policy with \textit{a simple act of explanation} by showing a message such as \textit{"The AI agent thinks you should view this problem as a Worked Example to learn how some new rules work."}. We compared this policy against two conditions, the Expert policy, and a student decision making policy. Our results show that 1) the Batch DRL policy with explanations significantly improved student learning performance more than the Expert policy; and 2) no significant differences were found between the Expert policy and student decision making. Overall, our results suggest that \textit{pairing simple explanations with the Batch DRL policy} can be an important and effective technique for applying RL to real-life, human-centric tasks. 

Our goal in this work is to build effective yet robust models to predict unreliable and inconsistent in-kind donations at both weekly and monthly levels for two food banks across coasts: the Food Bank of Central Eastern North Carolina in North Carolina and Los Angeles Regional Food Bank in California. We explore three factors: model, data length, and window type. For the model, we evaluate a series of classic time-series forecasting models against the state-of-the-art approaches such as Bayesian Structural Time Series modeling (BSTS) and deep learning models; for the data length, we vary training data from 2 weeks to 13 years; for the window type, we compare sliding vs. expanding. Our results show the effectiveness of different models heavily depends on the data length and the window type as well as characteristics of the food bank. Motivated by these findings, we investigate the effectiveness of employing an average of all predictions formed by considering all three factors at both monthly and weekly levels for both food banks. Our results show that this average of predictions significantly and consistently outperforms all classical models, deep learning, and BSTS for the donation prediction at both monthly and weekly levels for both food banks. 

It is shown that appropriate therapeutic management at early stages of sepsis are crucial for preventing further deterioration and irreversible organ damage. Although previous studies considered the cellular and physiological responses as the components of sepsis-related predictive models, temporal connections among the responses have not been widely studied. The objective of this study is to investigate simultaneous changes in cellular and physiological responses represented by 16 clinical variables contributing to seven organ system dysfunctions in patients with sepsis to predict in-hospital mortality. Organ dysfunctions were represented by undirected weighted network models composed of: i) nodes (i.e., 16 clinical variables and three biomarkers including procalcitonin, C-reactive protein, and sedimentation rate), ii) edges (i.e., connection between pair of nodes representing simultaneous dysfunctions), and iii) weights representing the persistence of the co-occurrence of two dysfunctions. Data was collected from 13,367 adult patients (corresponding to 17,953 visits) admitted to the study hospital from July 1, 2013, to December 31, 2015. The study population were categorized based on clinical criteria representing sepsis progression to identify different subpopulations. The findings quantify the optimal window for defining the simultaneity of two dysfunctions, the network properties corresponding to different subpopulations, the discriminatory patterns of simultaneous dysfunctions among subpopulations and in-hospital mortality prediction. The results show that the level of persistence of simultaneous dysfunctions are subpopulation-specific. Insights from this study regarding optimal thresholds of the persistence and combination of simultaneous organ dysfunctions can inform policies to personalize the in-hospital mortality prediction.